Multiple Media Correlation: Theory and Applications

This thesis introduces multiple media correlation, a new technology for the automatic alignment of multiple media objects such as text, audio, and video. This research began with the question: what can be learned when multiple multimedia components are analyzed simultaneously? Most ongoing research in computational multimedia has focused on queries, indexing, and retrieval within a single media type. Video is compressed and searched independently of audio, text is indexed without regard to temporal relationships it may have to other media data. Multiple media correlation provides a framework for locating and exploiting correlations between multiple, potentially heterogeneous, media streams. The goal is computed synchronization, the determination of temporal and spatial alignments that optimize a correlation function and indicate commonality and synchronization between media objects. The model also provides a basis for comparison of media in unrelated domains. There are many real-world applications for this technology, including speaker localization, musical score alignment, and degraded media realignment. Two applications, text-to-speech alignment and parallel text alignment, are described in detail with experimental validation. Text-to-speech alignment computes the alignment between a textual transcript and speech-based audio. The presented solutions are effective for a wide variety of content and are useful not only for retrieval of content, but in support of automatic captioning of movies and video. Parallel text alignment provides a tool for the comparison of alternative translations of the same document that is particularly useful to the classics scholar interested in comparing translation techniques or styles. The results presented in this thesis include (a) new media models more useful in analysis applications, (b) a theoretical model for multiple media correlation, (c) two practical application solutions that have wide-spread applicability, and (d) Xtrieve, a multimedia database retrieval system that demonstrates this new technology and demonstrates application of multiple media correlation to information retrieval. This thesis demonstrates that computed alignment of media objects is practical and can provide immediate solutions to many information retrieval and content presentation problems. It also introduces a new area for research in media data analysis

High Quality Alias Free Image Rotation

This paper presents new algorithms for the rotation of images. The primary design criteria for these algorithms is very high quality. Common methods for image rotation, including convolutional and separable approaches, are examined and shown to exhibit significant high frequency aliasing problems. A new resampling filter design methodology is presented which minimizes the problem for conventional convolution-based image rotation. The paper also presents a new separable image rotation algorithm which exhibits improved performance in term of reduction in artifacts and an efficient $O(N^{2} log N)$ running time

Multimedia Data Analysis using ImageTcl (Extended version)

ImageTcl is an new system which provides powerful Tcl/Tk based media scripting capabilities similar to those of the ViewSystem and Rivl in a unique environment that allows rapid prototyping and development of new components in the C++ language. Powerful user tools automate the creation of new components as well as the addition of new data types and file formats. Applications using ImageTcl at the Dartmouth Experimental Visualization Laboratory (DEVLAB) include multiple stream media data analysis, automatic image annotation, and image sequence motion analysis. ImageTcl combines the high speed of compiled languages with the testing and parameterization advantages of scripting languages

Multiple Media Stream Data Analysis: Theory and Applications (Extended version)

This paper presents a new model for multiple media stream data analysis as well as descriptions of some applications of this model in development at Dartmouth College. This model formalizes the exploitation of correlations between multiple, potentially heterogeneous, media streams in support of numerous application areas. The goal of the technique is to determine temporal and spatial alignments which optimize a correlation function and indicate commonality and synchronization between media streams. It also provides a framework for comparison of media in unrelated domains. Applications such as text-to-speech alignment, functional magnetic resonance imaging, speaker localization, and degraded media realignment are described

Multimedia authoring, development environments, and digital video editing

Multimedia systems integrate text, audio, video, graphics, and other media and allow them to be utilized in a combined and interactive manner. Using this exciting and rapidly developing technology, multimedia applications can provide extensive benefits in a variety of arenas, including research, education, medicine, and commerce. While there are many commercial multimedia development packages, the easy and fast creation of a useful, full-featured multimedia document is not yet a straightforward task. This paper addresses issues in the development of multimedia documents, ranging from user-interface tools that manipulate multimedia documents to multimedia communication technologies such as compression, digital video editing and information retrieval. It outlines the basic steps in the multimedia authoring process and some of the requirements that need to be met by multimedia development environments. It also presents the role of video, an essential component of multimedia systems and the role of programming in digital video editing. A model is described for remote access of distributed video. The paper concludes with a discussion of future research directions and new uses of multimedia documents

Episodic memory function is associated with multiple measures of white matter integrity in cognitive aging

Previous neuroimaging research indicates that white matter injury and integrity, measured respectively by white matter hyperintensities (WMH) and fractional anisotropy (FA) obtained from diffusion tensor imaging (DTI), differ with aging and cerebrovascular disease (CVD) and are associated with episodic memory deficits in cognitively normal older adults. However, knowledge about tract-specific relationships between WMH, FA, and episodic memory in aging remains limited. We hypothesized that white matter connections between frontal cortex and subcortical structures as well as connections between frontal and temporo-parietal cortex would be most affected. In the current study, we examined relationships between WMH, FA and episodic memory in 15 young adults, 13 elders with minimal WMH and 15 elders with extensive WMH, using an episodic recognition memory test for object-color associations. Voxel-based statistics were used to identify voxel clusters where white matter measures were specifically associated with variations in episodic memory performance, and white matter tracts intersecting these clusters were analyzed to examine white matter-memory relationships. White matter injury and integrity measures were significantly associated with episodic memory in extensive regions of white matter, located predominantly in frontal, parietal, and subcortical regions. Template based tractography indicated that white matter injury, as measured by WMH, in the uncinate and inferior longitudinal fasciculi were significantly negatively associated with episodic memory performance. Other tracts such as thalamo-frontal projections, superior longitudinal fasciculus, and dorsal cingulum bundle demonstrated strong negative associations as well. The results suggest that white matter injury to multiple pathways, including connections of frontal and temporal cortex and frontal-subcortical white matter tracts, plays a critical role in memory differences seen in older individuals

The Potential Impact of Nuclear Conflict on Ocean Acidification

We demonstrate that the global cooling resulting from a range of nuclear conflict scenarios would temporarily increase the pH in the surface ocean by up to 0.06 units over a 5-year period, briefly alleviating the decline in pH associated with ocean acidification. Conversely, the global cooling dissolves atmospheric carbon into the upper ocean, driving a 0.1 to 0.3 unit decrease in the aragonite saturation state (Ωarag) that persists for ∼10 years. The peak anomaly in pH occurs 2 years post conflict, while the Ωarag anomaly peaks 4- to 5-years post conflict. The decrease in Ωarag would exacerbate a primary threat of ocean acidification: the inability of marine calcifying organisms to maintain their shells/skeletons in a corrosive environment. Our results are based on sensitivity simulations conducted with a state-of-the-art Earth system model integrated under various black carbon (soot) external forcings. Our findings suggest that regional nuclear conflict may have ramifications for global ocean acidification

Rapidly expanding nuclear arsenals in Pakistan and India portend regional and global catastrophe

Pakistan and India may have 400 to 500 nuclear weapons by 2025 with yields from tested 12- to 45-kt values to a few hundred kilotons. If India uses 100 strategic weapons to attack urban centers and Pakistan uses 150, fatalities could reach 50 to 125 million people, and nuclear-ignited fires could release 16 to 36 Tg of black carbon in smoke, depending on yield. The smoke will rise into the upper troposphere, be self-lofted into the stratosphere, and spread globally within weeks. Surface sunlight will decline by 20 to 35%, cooling the global surface by 2° to 5°C and reducing precipitation by 15 to 30%, with larger regional impacts. Recovery takes more than 10 years. Net primary productivity declines 15 to 30% on land and 5 to 15% in oceans threatening mass starvation and additional worldwide collateral fatalities

Functional Redundancy of Class I Phosphoinositide 3-Kinase (PI3K) Isoforms in Signaling Growth Factor-Mediated Human Neutrophil Survival

We have investigated the contribution of individual phosphoinositide 3-kinase (PI3K) Class I isoforms to the regulation of neutrophil survival using (i) a panel of commercially available small molecule isoform-selective PI3K Class I inhibitors, (ii) novel inhibitors, which target single or multiple Class I isoforms (PI3Kα, PI3Kβ, PI3Kδ, and PI3Kγ), and (iii) transgenic mice lacking functional PI3K isoforms (p110δKOγKO or p110γKO). Our data suggest that there is considerable functional redundancy amongst Class I PI3Ks (both Class IA and Class IB) with regard to GM-CSF-mediated suppression of neutrophil apoptosis. Hence pharmacological inhibition of any 3 or more PI3K isoforms was required to block the GM-CSF survival response in human neutrophils, with inhibition of individual or any two isoforms having little or no effect. Likewise, isolated blood neutrophils derived from double knockout PI3K p110δKOγKO mice underwent normal time-dependent constitutive apoptosis and displayed identical GM-CSF mediated survival to wild type cells, but were sensitized to pharmacological inhibition of the remaining PI3K isoforms. Surprisingly, the pro-survival neutrophil phenotype observed in patients with an acute exacerbation of chronic obstructive pulmonary disease (COPD) was resilient to inactivation of the PI3K pathway